Entrustable Professional Activities Research Articles

Design of an instrument to evaluate entrustable professional activities for general surgery

Design of an instrument to evaluate entrustable professional activities for general surgery

Rethinking Informed Consent as an EPA in Surgery.

Over the past decade, entrustable professional activities (EPAs) have become an important element in the competency-based medical education movement. In this Commentary, the authors explore informed consent as an EPA within resident surgical training. In doing so, they foreground the concept of culture and reexamine the nature of trust and entrustment decisions from within a cultural framework. The authors identify role modeling and professional identity formation as core elements in the training process and suggest that faculty are sometimes better off using these tools than uncritically adopting a formal EPA framework for what is, in essence, a professionally oriented and values-based moral enterprise. They conclude that EPAs work best when they are developed at a local level, stressing the unique specialty and program culture as well as the care that must be taken when attempting to transfer notions of entrustment from the undergraduate medical education level to graduate medical education settings.

A Modified Delphi Consensus Approach to Define Entrustable Professional Activities for Neurocritical Care Advanced Practice Providers (S39.005)

A Modified Delphi Consensus Approach to Define Entrustable Professional Activities for Neurocritical Care Advanced Practice Providers (S39.005)

Educating Trainees to Treat Peripheral Arterial Disease: Challenges and Opportunities

Educating trainees to treat Peripheral Artery Disease (PAD) carries specific contemporary challenges. The national increase of the prevalence of PAD combined with a significant shortage of vascular surgeons creates a need for concern for future management of this complex disease. Over the past two decades, traditional (5+2) and integrated (0+5) paradigms have fostered trainee annual growth and comparable case distribution and volumes in endovascular and open surgical treatment options have been maintained. Close evaluation into not only the absolute numbers of surgical cases, but the level of trainee involvement in each logged case is recommended. Future implementation of the Entrustable Professional Activity (EPA) modules will hopefully assist in ensuring linear development of surgical skill and judgement. Additionally, advances in individual and systems level techniques to enhance skill acquisition in the form of “off-the job training” and simulation-based training may provide an enhancement to traditional technical training methods. Finally, the possibility and role of Artificial Intelligence in vascular surgery skill training must not be ignored, but carefully explored and utilized to modernize cognitive and technical skill preparation for trainees in the and delivery of care for PAD patients. Overall, the training residents for the treatment of PAD patients will be associated with new challenges that vascular surgery must embrace and surmount to advance our specialty.

Transitioning From AFMRD Entrustable Professional Activities to ABFM Core Outcomes to Measure Clinical Preparedness.

Transitioning From AFMRD Entrustable Professional Activities to ABFM Core Outcomes to Measure Clinical Preparedness.

Authors' Response to "Transitioning From AFMRD Entrustable Professional Activities to ABFM Core Outcomes to Measure Clinical Preparedness".

Authors' Response to "Transitioning From AFMRD Entrustable Professional Activities to ABFM Core Outcomes to Measure Clinical Preparedness".

Motivations for Entrustable Professional Activity Assessment: Gaps Between Curriculum Theory and Resident Reality

Background Previous research demonstrates mixed reactions from residents toward competency-based medical education (CBME), and entrustable professional activities (EPAs) specifically. However, understanding what motivates residents to obtain EPAs may be vital to the longevity of CBME, given the emphasis on assessment for learning under this paradigm. Objective This study explored resident perspectives across 3 domains: motivation for obtaining EPAs, perceived importance of EPAs, and overall thoughts on CBME curriculum. Methods This was a sequential exploratory mixed-methods study involving 2 phases of data collection. Phase 1 was semi-structured interviews with residents enrolled in CBME at one Canadian institution from November 2019 to July 2020. Analyses included thematic and manifest content analysis. Phase 2 was an electronic close-ended survey to capture residents’ primary motivation for requesting EPAs and importance of EPAs for learning. Survey data were analyzed descriptively. Results Of 120 eligible residents, 25 (21%) and 107 (89%) participated in the interview and survey, respectively. Program requirement was the dominant motivation for obtaining EPAs. There was variability in perceived importance of EPAs on learning. Increased resident workload, gaming the system to maximize EPA scores, and lack of shared ownership from preceptors were cited as critiques of the curriculum. Survey responses corroborated interview findings. Conclusions Although many residents recognize the value of EPAs, the majority are not intrinsically motivated to seek out assessment under the current CBME framework.

Effects of game-based learning integrated with the self-regulated learning strategy on nursing students' entrustable professional activities: A quasi-experimental study

BackgroundIn the field of nursing education, the expertise and knowledge required to perform Leopold's Maneuvers are critical, forming an integral part of Entrustable Professional Activities for nursing students. As a result, mastering Leopold's Maneuvers has become a core component of nursing education. Despite this, the prevailing didactic methods in many nursing courses tend to limit interactive and contextual learning experiences, which can hinder students' ability to engage deeply with the subject matter. This lack of engagement may subsequently affect both the effectiveness of students' learning and students' clinical judgment capacity, essential elements in the competencies of Entrustable Professional Activities. MethodsA quasi-experimental design was employed in this study, involving a total of 55 nursing university students, with 28 participants in the experimental group and 27 in the control group. The research incorporated game-based learning integrated with the self-regulated learning strategy, and employed a pretest, posttest, and questionnaires to assess students' learning performance, self-efficacy, and learning motivation. The experimental group participated in game-based learning with the self-regulated learning strategy, whereas the control group experienced game-based learning without the integration of self-regulated learning. Learning performance was analyzed using an independent samples t-test, while Analysis of Covariance was employed to compare the self-efficacy and learning motivation of the two groups. ResultsThe findings showed that the integration of game-based learning with the self-regulated learning strategy can significantly improve students' learning performance, self-efficacy, and learning motivation. ConclusionAs an integral part of Entrustable Professional Activities, Leopold's Maneuvers were integrated into an interactive game-based learning environment, coupled with the self-regulated learning strategy in nursing education. This approach aimed at cultivating core competencies among nursing students. This study has demonstrated its efficacy in engaging students effectively, serving as a dynamic resource for immersive learning in Leopold's Maneuvers, aligning well with the design considerations for promoting Entrustable Professional Activities in nursing education.

LEVEL OF SUPERVISION FOR THE EPAS COMMON TO GENERAL PEDIATRICS AND THE SUBSPECIALTIES DECREASES FROM RESIDENCY TO FELLOWSHIP

ObjectivesTo compare level of supervision (LOS) ratings of graduating pediatric residents with their assessments as fellows for the 5 Entrustable Professional Activities (EPAs) common to general pediatrics and the subspecialties and to determine if the difference between ratings from residency to fellowship is less for the QI and PRACTICE MANAGEMENT EPAs, since the skills needed to perform these may be less context-dependent. MethodsWe compared ratings of graduating residents with their assessments as fellows using LOS data from 2 sequential EPA studies. ResultsThere were 65 ratings from 41 residents at the 1st fellow assessment. At graduation, most residents needed little to no supervision for all EPAs with 94% (61/65) of ratings level 4 or 5. In contrast, only 5/65 (8%) of the 1st fellow assessments were level 4 or 5. The ratings difference for the QI and PRACTICE MANAGEMENT EPAs was similar to the others. ConclusionsLOS ratings for the EPAs common to generalists and subspecialists reset as residents become fellows. There was no evidence that the QI and PRACTICE MANAGEMENT EPAs are less context-dependent. This study provides additional validity evidence for using these LOS scales to assess trainees in pediatric residency and fellowship.

The Next Era of Assessment: Can Ensuring High-Quality, Equitable Patient Care Be the Defining Characteristic?

Previous eras of assessment in medical education have been defined by how assessment is done, from knowledge exams popularized in the 1960s to the emergence of work-based assessment in the 1990s to current efforts to integrate multiple types and sources of performance data through programmatic assessment. Each of these eras was a response to why assessment was performed (e.g., assessing medical knowledge with exams; assessing communication, professionalism, and systems competencies with work-based assessment). Despite the evolution of assessment eras, current evidence highlights the graduation of trainees with foundational gaps in the ability to provide high-quality care to patients presenting with common problems, and training program leaders report they graduate trainees they would not trust to care for themselves or their loved ones. In this article, the authors argue that the next era of assessment should be defined by why assessment is done: to ensure high-quality, equitable care. Assessment should place focus on demanding graduates possess the knowledge, skills, attitudes, and adaptive expertise to meet the needs of all patients and ensuring that graduates are able to do this in an equitable fashion. The authors explore 2 patient-focused assessment approaches that could help realize the promise of this envisioned era: entrustable professional activities (EPAs) and resident sensitive quality measures (RSQMs)/TRainee Attributable and Automatable Care Evaluations in Real-time (TRACERs). These examples illustrate how the envisioned next era of assessment can leverage existing and new data to provide precision education assessment that focuses on providing formative and summative feedback to trainees in a manner that seeks to ensure their learning outcomes prepare them to ensure high-quality, equitable patient outcomes.

Mobile Application to Improve Just-in-Time 2-Way Formative Feedback in Graduate Medical Education

Background An easy-to-use application to facilitate direct observation and allow for 2-way feedback between residents and faculty is needed. Objective To develop a mobile-based application (app) with the goals of (1) providing just-in-time feedback to residents; (2) improving timeliness of feedback by faculty; and (3) allowing residents to comment on the value of faculty feedback. Methods Fifty-one of 69 (74%) internal medicine (IM) residents and 20 of 25 (80%) IM core faculty participated in the study from July 1, 2020, to December 31, 2021. An iOS app was designed by authors with expertise in medical education and application development to capture entrustable professional activities (EPAs)-based feedback (eg, informed consent) based on direct observation of residents’ skills in the workplace. App utilization and narrative feedback characteristics of faculty comments were examined by exporting the data from the database server. The end user satisfaction was examined using a survey instrument. Results Eighty-seven percent of assessments (117 of 134) initiated were fully completed by residents and faculty. Faculty narrative comments were noted in 97% (114 of 117) of completed assessments and 64% (75 of 117) of residents’ feedback to the faculty contained narrative comments. Eighty-three percent (97 of 117) of comments were behaviorally specific and 71% (83 of 117) contained an actionable item. Eighty-six percent (18 of 21) of residents and 90% (9 of 10) of core faculty stated that this application promoted an educational interaction between them. Conclusions This app facilitates the efficient completion of EPA-based formative assessments and captures bidirectional feedback in the workplace setting.

Development of entrustable professional activities for family medicine in South Africa

South Africa is undergoing a significant shift towards implementing enhanced workplace-based assessment methodologies across various specialist training programmes, including family medicine. This paradigm involves the evaluation of Entrustable Professional Activities (EPAs) through comprehensive portfolios of evidence, which a local and national clinical competency committee then assesses. The initial phase of this transformative journey entails the meticulous development of EPAs rooted in discrete units of work. Each EPA delineates the registrar’s level of entrustment for autonomous practice, along with the specific supervision requirements. This concise report details the collaborative effort within the discipline of family medicine in South Africa, culminating in the consensus formation of 22 meticulously crafted EPAs for postgraduate family medicine training. The article intricately outlines the systematic structuring and rationale behind the EPAs, elucidating the iterative process employed in their development. Notably, this marks a groundbreaking milestone as the first comprehensive documentation of EPAs nationally for family medicine training in Africa.

Comparison of OBGYN postgraduate curricula and assessment methods between Canada and the Netherlands: an auto-ethnographic study.

Although the Dutch and the Canadian postgraduate Obstetrics and Gynecology (OBGYN) medical education systems are similar in their foundations [programmatic assessment, competency based, involving CanMED roles and EPAs (entrustable professional activities)] and comparable in healthcare outcome, their program structures and assessment methods considerably differ. We compared both countries' postgraduate educational blueprints and used an auto-ethnographic method to gain insight in the effects of training program structure and assessment methods on how trainees work. The research questions for this study are as follows: what are the differences in program structure and assessment program in Obstetrics and Gynecology postgraduate medical education in the Netherlands and Canada? And how does this impact the advancement to higher competency for the postgraduate trainee? We found four main differences. The first two differences are the duration of training and the number of EPAs defined in the curricula. However, the most significant difference is the way EPAs are entrusted. In Canada, supervision is given regardless of EPA competence, whereas in the Netherlands, being competent means being entrusted, resulting in meaningful and practical independence in the workplace. Another difference is that Canadian OBGYN trainees have to pass a summative written and oral exit examination. This difference in the assessment program is largely explained by cultural and legal aspects of postgraduate training, leading to differences in licensing practice. Despite the fact that programmatic assessment is the foundation for assessment in medical education in both Canada and the Netherlands, the significance of entrustment differs. Trainees struggle to differentiate between formative and summative assessments. The trainees experience both formative and summative forms of assessment as a judgement of their competence and progress. Based on this auto-ethnographic study, the potential for further harmonization of the OBGYN PGME in Canada and the Netherlands remains limited.

Psychometric Properties of Entrustable Professional Activity-Based Objective Structured Clinical Examinations During Transition From Undergraduate to Graduate Medical Education: A Generalizability Study.

The objective structured clinical examination (OSCE) assesses clinical competence in health sciences education. There is little research regarding the reliability and validity of using an OSCE during the transition from undergraduate to graduate medical education. The goal of this study was to measure the reliability of a unique 2-rater entrustable professional activity (EPA)-based OSCE format for transition to internship using generalizability theory for estimating reliability. During the 2018 to 2022 academic years, 5 cohorts of interns (n = 230) at the University of Iowa Hospital and Clinics participated in a 6-station OSCE assessment delivered during orientation. A univariate and multivariate generalizability study (G study) was conducted on the scores generated by the 3 cases in the orientation OSCE that shared the 2-rater format. This analysis was supplemented with an associated decision study (D study). The univariate G study for the cases that used a simulated patient and a faculty rater demonstrated that this OSCE generated a moderately reliable score with 3 cases. The D study showed that increasing the OCSE to 12 cases yielded a mean score reliable enough (G = 0.76) for making high-stakes normative decisions regarding remediation and readiness to practice. The universe score correlation between 2 types of raters was 0.398. The faculty ratings displayed a larger proportion of universe (true) score variance and yielded a more reliable (G = 0.433) score compared with the standardized patient ratings (G = 0.337). This study provides insight into the development of an EPA-based OSCE. The univariate G study demonstrated that when using the 2 rater types, this assessment could generate a moderately reliable score with 3 cases. The multivariate G study showed that the 2 types of raters assessed different aspects of clinical skills and faculty raters were more reliable.

Better together: A multistakeholder approach to developing specialty-wide entrustable professional activities in emergency medicine.

Entrustable professional activities (EPAs) are a widely used framework for curriculum and assessment, yet the variability in emergency medicine (EM) training programs mandates the development of EPAs that meet the needs of the specialty as a whole. This requires eliciting and incorporating the perspectives of multiple stakeholders (i.e., faculty, residents, and patients) in the development of EPAs. Without a shared understanding of what a resident must be able to do upon graduation, we run the risk of advancing ill-prepared residents that may provide inconsistent care. In an effort to address these challenges, beginning in February 2020, the authors assembled an advisory board of 25 EM faculty to draft and reach consensus on a final list of EPAs that can be used across all training programs within the specialty of EM. Using modified Delphi methodology, the authors came to consensus on an initial list of 22 EPAs. The authors presented these EPAs to faculty supervisors, residents, and patients for refinement. The authors collated and analyzed feedback from focus groups of residents and patients using thematic analysis. The EPAs were subsequently refined based on this feedback. Stakeholders in EM residency training endorsed a final revised list of 22 EPAs. Stakeholder focus groups highlighted two main thematic considerations that helped shape the finalized list of EM EPAs: attention to the meaningful nuances of EPA language and contextualizing the EPAs and viewing them developmentally. To foreground all key stakeholders within the EPA process for EM, the authors chose within the development process to draft; come to consensus; and refine EPAs for EM in collaboration with relevant faculty, patient, and resident stakeholders. Each stakeholder group contributed meaningfully to the content and intended implementation of the EPAs. This process may serve as a model for others in developing stakeholder-responsive EPAs.

Analysis of fellowship program director opinions of entrustable professional activities in adolescent medicine fellowship.

This study aimed to explore the minimum entrustable professional activity (EPA) supervision levels at which pediatric fellowship program directors (FPDs) would be willing to graduate fellows and the levels deemed necessary for safe and effective practice for each of the common pediatric subspecialty and the four adolescent medicine-specific EPAs. This cross-sectional study utilized survey data from pediatric FPDs in 2017. FPDs indicated the minimum level of supervision (LOS) for fellows at graduation and for safe and effective practice. 82 percent (23/28) of adolescent medicine FPDs completed the survey. For each EPA, there were differences (p<0.05) between LOS expected for graduation and for safe and effective practice. There was also variability in the level at which FPDs would graduate fellows. This study summarizes pediatric FPD opinions regarding the minimum levels of supervision required for fellows at the time of graduation as well as the levels deemed necessary for safe and effective practice. The difference between the minimum LOS at which FPDs would graduate a fellow and that deemed appropriate for safe and effective practice, along with variability in minimum LOS for graduation, highlight the need for clearer standards for fellowship graduation as well as more structured early career support for ongoing learning. These data highlight variability in FPD opinion regarding such expectations and both the need to better define desired training outcomes and potential need for post-graduation supervision in clinical practice.

In EPAs we trust, is quality and safety a must? A cross-specialty analysis of entrustable professional activity guides

Purpose The inclusion of quality improvement (QI) and patient safety (PS) into CanMEDS reflects an expectation that graduating physicians are competent in these areas upon training completion. To ensure that Canadian postgraduate specialty training achieves this, the translation of QI/PS competencies into training standards as part of the implementation of competency-based medical education requires special attention. Methods We conducted a cross-specialty, multi-method analysis to examine how QI/PS was incorporated into the EPA Guides across 11 postgraduate specialties in Canada. Results We identify cross-specialty variability in how QI/PS is incorporated, positioned, and emphasized in EPAs and milestones. QI/PS was primarily referenced alongside clinical activities rather than as a sole competency or discrete activity. Patterns were characterized in how QI/PS became incorporated into milestones through repetition and customization. QI/PS was also decoupled, conceptualized, and emphasized differently across specialties. Conclusions Variability in the inclusion of QI/PS in EPAs and milestones has important implications considering the visibility and influence of EPA Guides in practice. As specialties revisit and revise EPA Guides, there is a need to balance the standardization of foundational QI/PS concepts to foster shared understanding while simultaneously ensuring context-sensitive applications across specialties. Beyond QI/PS, this study illuminates the challenges and opportunities that lie in bridging theoretical frameworks with practical implementation in medical education, prompting broader consideration of how intrinsic roles and emergent areas are effectively incorporated into competency-based medical education.

Validity of a Novel Entrustable Professional Activities Based End of Shift Assessment

Validity of a Novel Entrustable Professional Activities Based End of Shift Assessment

An Analysis of Workplace-Based Assessments for Core Entrustable Professional Activities for Entering Residency: Does Type of Clinical Assessor Influence Level of Supervision Ratings?

The authors describe use of the workplace-based assessment (WBA) coactivity scale according to entrustable professional activities (EPAs) and assessor type to examine how diverse assessors rate medical students using WBAs. A WBA data collection system was launched at Oregon Health and Science University to visualize learner competency in various clinical settings to foster EPA assessment. WBA data from January 14 to June 18, 2021, for medical students (all years) were analyzed. The outcome variable was level of supervisor involvement in each EPA, and the independent variable was assessor type. A total of 7,809 WBAs were included. Most fourth-, third-, and second-year students were assessed by residents or fellows (755 [49.5%], 1,686 [48.5%], and 918 [49.9%], respectively) and first-year students by attending physicians (803 [83.0%]; P < .001). Attendings were least likely to use the highest rating of 4 (1 was available just in case; 2,148 [56.7%] vs 2,368 [67.7%] for residents; P < .001). Learners more commonly sought WBAs from attendings for EPA 2 (prioritize differential diagnosis), EPA 5 (document clinical encounter), EPA 6 (provide oral presentation), EPA 7 (form clinical questions and retrieve evidence-based medicine), and EPA 12 (perform general procedures of a physician). Residents and fellows were more likely to assess students on EPA 3 (recommend and interpret diagnostic and screening tests), EPA 4 (enter and discuss orders and prescriptions), EPA 8 (give and receive patient handover for transitions in care), EPA 9 (collaborate as member of interprofessional team), EPA 10 (recognize and manage patient in need of urgent care), and EPA 11 (obtain informed consent). Learners preferentially sought resident vs attending supervisors for different EPA assessments. Future research should investigate why learners seek different assessors more frequently for various EPAs and if assessor type variability in WBA levels holds true across institutions.

A Modified Delphi Consensus Approach to Define Entrustable Professional Activities for Neurocritical Care Advanced Practice Providers.

To define consensus entrustable professional activities (EPAs) for neurocritical care (NCC) advanced practice providers (APPs), establish validity evidence for the EPAs, and evaluate factors that inform entrustment expectations of NCC APP supervisors. A three-round modified Delphi consensus process followed by application of the EQual rubric and assessment of generalizability by clinicians not affiliated with academic medical centers. Electronic surveys. NCC APPs (n = 18) and physicians (n = 12) in the United States with experience in education scholarship or APP program leadership. None. The steering committee generated an initial list of 61 possible EPAs. The panel proposed 30 additional EPAs. A total of 47 unique nested EPAs were retained by consensus opinion. The steering committee defined six core EPAs addressing medical knowledge, procedural competencies, and communication proficiency which encompassed the nested EPAs. All core EPAs were retained and subsequently met the previously described cut score for quality and structure using the EQual rubric. Most clinicians who were not affiliated with academic medical centers rated each of the six core EPAs as very important or mandatory. Entrustment expectations did not vary by prespecified groups. Expert consensus was used to create EPAs for NCC APPs that reached a predefined quality standard and were important to most clinicians in different practice settings. We did not identify variables that significantly predicted entrustment expectations. These EPAs may aid in curricular design for an EPA-based assessment of new NCC APPs and may inform the development of EPAs for APPs in other critical care subspecialties.